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13 ,$. Department of Energy j 



Electricity Delivery an# | 

ELECTRIC EMERGENCY INCIDENT AND 

Approval Expires: 05/31/2021 v . - J 

Energy Reliability 

Form. OE-417 | 

DISTURBANCE REPORT 



NOTICE: Hus report is mandatory under Public Law 93-275. Failure to comply may result in criminal fines, civil penalties and other sanctions as provided by 
law. For the sanctions and the provisions concerning the confidentiality of information submitted on this form, see General Information portion of the instructions. 
Title 18 USC 1001 makes it a criminal offense for any person knowingly and willingly to make to any Agency or Department of the United States any false, 
fictitious, or fraudulent statements as tu any matter within its jurisdiction. 


RESPONSE DUE: 

Within 1 hour of the incident, submit Schedule 1 and lines M - Q in Schedule 2 as an Emergency Alert report if criteria 1-8 are met. 

Within 6 hours of the incident, submit Schedule 1 and lines M - Q in Schedule 2 as a Normal Report if only criteria 9-12 are met. 

By the later of 24 hours after the recognition of the incident OR by the end of the next business day submit Schedule 1 & lines M - Q in Schedule 2 as a System 
Report if criteria 13-24 are met. Note 4 00pm local time will be considered the end of the business day 

Submit updates as needed and/or a final report (all of Schedules 1 and 2) within 72 hours of the incident. 

For NERC reporting entities registered in the United States; NERC has approved that the form OE-417 meets the submittal requirements for NERC. There may 
be other applicable regional, state and local reporting requirements. 


METHODS OF FILING RESPONSE 
(Retain a completed copy of this form for your files.) 

Online: Submit form via online submission at: https://wurw.oe.netl.doe.gov/OE417/ 

FAX: FAX Form OE-417 to the following facsimile number (202) 586-8485. 

Alternate: If you are unable to submit online or by fax, forms may be e-mailed to doefaaeoc@Uo.doe.gov. or call and report the information to the 

_following telephone number (202) 586-8100.__ 


SCHEDULE 1 ~ ALERT CRITERIA 

_ (Page i of 4) _ 


Criteria for Filing (Check all that apply) 
See Instructions For Mure Information 


EMERGENCY ALERT 
File within 1-Hour 

If any box 1-8 on the right is 
checked, this form must be 
filed within l hour of the 
incident; check Emergency 
Alert (for the Alert Status) on 
Line A below. 


1. [ ] Physical attack that causes major interruptions or impacts to critical infrastructure facilities or to operations 

2. [X3 Cyber event that causes interruptions of electrical system operations 

3. [ ] Complete operational failure or shut-down of the trans m ission and/or distribution electrical system 

4. [ ] Electrical System Separation (Islanding) where part or parts of a power grid remain(s) operational in an otherwise 

blacked out area or within the partial failure of an integrated electrical system 

5. [ 1 Uncontrolled loss of 300 Megawatts or more of firm system loads for 15 minutes or more from a single 

incident 

6. [ ] Firm load shedding of 100 Megawatts or more implemented under emergency operational policy 

7. [ ] System-wide voltage reductions of 3 percent or more 

8. ( ] Public appeal to reduce the use of electricity for purposes of maintaining the continuity of the Bulk Electric System 


NORMAL REPORT 
File within 6-Hours 

If any box 9-12 on the right is 
checked AND none of the 
boxes 1-8 arc checked, this 
form must be filed within 6 
hours of the incident; check 
Normal Report (for the Alert 
Status) on Line A below. 


9. [ ] Physical attack that could potentially impact electric power system adequacy or reliability; ox vandalism which 

targets components of any security systems 

10. [X3 Cyber event that could potentially impact electric power system adequacy or reliability 

11. [ ] Loss of electric service to more than 50,000 customers for 1 hour or more 

12. [ ] Fuel supply emergencies that could impact electric power system adequacy or reliability 







SYSTEM REPORT 
File within l-Business Day 

If any box 13-24 on the right is 
checked AND none of the 
boxes 1-12 are checked, this 
form most be filed by the later 
of 24 hours after the 
recognition of the incident OR 
by the end of the next business 
day .Note 4:00pm local time 
will be considered die end of 
the business day. Check 
System Report (for the Alert 
Status) on Line A below. 


SCHEDULE 1 - ALERT CRITERIA - CONTINUED 

(Page 2 of 4) 


13. [ ] Damage or destruction of a Facility within its Reliability Coordinator Atm, Bal an ci n g Authority Area or 

Transmission Operator Area that results in action(s) to avoid a Bulk Electric System Emergency. 

14. £ ] Damage or destruction of its Facility that results from actual or suspected intentional human action. 

15. [ ] Physical threat to its Facility excluding weather or natural disaster related threats, which has the potential to 

degrade the normal operation of the Facility. Or suspicious device or activity at its Facility. 

16. [ ] Physical threat to its Bulk Electric System control center, excluding weather or natural disaster related threats, which 

has the potential to degrade the normal operation of the control center. Or suspicious device or activity at its Bulk 
Electric System control center. 

17. [ J Bulk Electric System Emergency resulting in voltage deviation on a Facility; A voltage deviation equal to or 

greater than 10% of nominal voltage sustained for greater than or equal to 15 continuous minutes. 

18. [ ] Uncontrolled loss of200 Megawatts or more of firm system loads for 15 minutes or more from, a single incident for 

entities with previous year's peak demand less than or equal to 3,000 Megawatts 

19. [ ] Total generation loss, within one minute of: greater than or equal to 2,000 Megawatts in the Eastern or Western 

Interconnection or greater than or equal to 1,400 Megawatts in the ERCOT Interconnection. 

20. [ ] Complete loss of off-site power (LOOP) affecting a nuclear generating station per ihe Nuclear Plant Interface 

Requirements. 

21. [ ] Unexpected Transmission loss wi thin its area, contrary to design, of three or more Bulk Electric System 

Facilities caused by a common disturbance (excluding successful automatic reclosing). 

22. [ ] Unplanned evacuation from its Bulk Electric System control center facility for 30 continuous minutes or more. 

23. [ ] Complete loss of Interpersonal Communication and Alternative Interpersonal Communication capability affecting 

its staffed Bulk Electric System control center for 30 continuous minutes or more. 

24. ( ] Complete loss of monitoring or control capability at its staffed Bulk Electric System control center for 30 

continuous minutes or more. 


If significant changes have occurred after filin g the initial report, re-file the form with the changes and check Update (for the Alert Status) on Line A below. 
The form must be re-filed within 72 hours of the incident with the latest information and Final (Alert Status) checked on line A below, unless updated 


UK 


jBKg®# 





i 

Alert Status (check one) 

Emergency Alert 

CXI 

1 Hour 

Normal Report 
[ 1 

6 Hours 

System Report 
[ 1 

1 Business Day 

Update 
[ 1 

As required 

Final 
[ ] 

72 Hours 

B. 

Organization Name 

sPower 


2180 South 1300 East Suite 600 Salt Lake City Utah 84106 


c. 


Address of Principal Business Office 
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Approval Expires: 05/31/2021 
Burden Per Response: JL8 hours 


SCHEDULE 1 


ALERT NOTICE 

3 of 4) 



[ ] Eastern [ ] Central [X3 Mountain 

f 1 Pacific f 1 Alaska f ] Hawaii 



Did the incident/distiubance originate in your r i 

system/area? (check one) 1 


Estimate of Amount of Demand Involved 
(Peak Megawatts) 


Estimate of Number of Customers Affected 



No[ ] 

Unknown [X] 

Zero [X] 

Unknown [ ] 

ZemtXl 

Unknown [ ] 


J. Cause 


SCHEDULE 1 - TYPE OF EMERGENCY 

Check all that apply 


K. Impact 


L. Action Taken 


□ Unknown 

□ Physical attack 

□ Threat of physical attack 

□ Vandalism 

□ Theft 

□ Suspicious activity 

□ Cyber event (information technology) 

ES Cyber event (operational technology) 

□ Fuel supply emergencies, interruption, or 

deficiency 

□ Generator loss or failure not due to fuel supply 
interruption or deficiency or transmission 
failure 

□ Transmission equipment failure (not including 
substation or switchyard) 

□ Failure at high voltage substation or switchyard 

□ Weather or natural disaster 

□ Operator action(s) 

□ Other 

SS Additional Infonnation/Comnieats: 
tniSai assessment revealed that n firewall exploit was Sfieiy utiBzed to 
execute a denial oi service attack that caused the fWewaSs to reboot 
leading to an approximately 5 minute communteafiona outage 


□ None 

□ Control center loss, failure, or evacuation 

□ Loss or degradation of control center monitoring 
or communication systems 

□ Damage or destruction of a facility 

□ Electrical system separation (islanding) 

□ Complete operational failure or shutdown of the 
transmission and/or distribution system 

□ Major transmission system interruption (three or 
more BES elements) 

□ Major distribution system interruption 

□ Uncontrolled loss of 200 MW or more of Gnu 
system loads for 15 minutes or more 

□ Loss of electric service to more than 50,000 
customers for 1 hour or more 

□ System-wide voltage reductions or 3 percent or 
more 

□ Voltage deviation on an individual facility of 
>10% for 15 minutes or more 

□ Inadequate electric resources to serve load 

□ Generating capacity loss of 1,400 MW or more 

□ Generating capacity loss of2,000 MW or more 

□ Complete loss of off-site power to a nuclear 
generating station 

SI Other 

IS Additional hiformatiou/C omments: 

firewall reboots resulted in briet communications outages 
(approximately S minutes) between field devices at sites and between 
the sites and sPovrette Control Center 


□ None 

□ Shed Firm Load: Load shedding of 100 
MW or more implemented under 
emergency operational policy (manually 
or automatically via UFLS or remedial 
action scheme) 

□ Public appeal to reduce the use of 
electricity for the purpose of maintaining 
the continuity of the electric power 
system 

□ Implemented a warning, alert, or 
contingency plan 

□ Voltage reduction 

□ Shed Interruptible Load 

□ Repaired or restored 

□ Mitigation implemented 

SI Other 

S3 Additional Jnfonuation/Comments 

After teaming of the potential cause o( the reboot, sPower 

started testing and deployment of an update to remove the 

exploited vulnerability 














































U.S. Department of Energy 
Electricity Delivery and 
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Energy Reliability 

DISTURBANCE RE TOR T 

Form OE-417 



3/7/2019 5:52:00 PM Submitted to DOE 


OMB No. 1901-0288 
Approval Expires: 05/31/2021 
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SCHEDULE 2 - NARRATIVE DESCRIPTION 

(Page 4 of 4) 

Information an Schedule 2 will not be disclosed to the public to the extent that it satisfies the criteria for exemption under the Freedom of Information Act, e.g., 
exemptions for confidential commercial information and trade secrets, certain information that could endanger the physical safety of an individual, or 

information designated as Critical Energy Infrastructure Information. 


NAME OF OFFICIAL THAT SHOULD BE CONTACTED FOR FOLLOW-UP OR ANY ADDITIONAL INFORMATION 


Kg|g|jggg 


ME— 


Name _ 


■.'Tale • . .__ 


one Number 


FAXNumber 


E-mail Address 


Provide a description of the incident and actions taken to resolve it. Include as appropriate, the cause of the incident/distnrbance, change in frequency, 
mitigation actions taken, equipment damaged, critical infrastructures interrupted, effects on othersystems, and preliminary results from any 
investigations. Be sure to identify: the estimate restoration date, the name of any lost high voltage substations or switchyards, whether there was any 
electrical system separation (and if there were, what the islanding boundaries were)," and the name of the generators and voltage lines thatwere lost 
(shown by capacity type and voltage size grouping). If necessary; copy and attach additional sheets. Equivalent doc uments , cont ain i ng this information can 
be supplied to meet the requirement; this includes the NERC EOP-004 Disturbance Report. Along with the .filing of Schedule 2, a final (updated) Schedule 1 
needs to be filed. Check the Final hoi on line A for Alert Status on Schedule 1 and submit this and the completed Scheduled no later than 72 hours 
after detection that a criterion was met. 


R. Narrative: 

This information will be provided in a subsequent update after additional information gathering 


S. Estimated Restoration Date for ah Affected Customers 
Who Can Receive Power 




Pioneer, Beacon 4, ABSR, DSR1, DSR2, Beacon 1,.Elevation C, WABSRB, Bayshore A, Bayshore B, Bayshore G, and Sdverde. 


Select if you approve of all of the infonnatian provided on the .Form being submitted to the North Amenca Electric 
Reliability Corporation (NERC) and/or the Electricity Information Sharing and Analysis Center (E IS AC). 

NERC is an entity that is certified by the Federal Energy Regulatory Commission to establish and enforce reliability 
standards for the bulk power system but that is not part of the Federal Government This information would be 
submitted to help fulfill the respondent's requirements under NERC’s reliability standards. 

If approval is given to alert NERC and/or E-ISAC the Fonn will be emailed to systemawareness@nexc.net and/or 
operaUons@eisac.com when it is submitted to DOE. DOE is uot responsible for ensuring the receipt of these/entails;-: 

by NERC and/or E ISAC. 

H Notify NERC | 8 Notify E ISAC . 
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OMB No. 1901 0288 


Borden Per Response: 1.8 hours 


NOTICE: This report is mandatory under Public Law 93-275. Failure to comply may result in criminal fines, civil penalties and other sanctions as provided by 
law. For the sanctions and the provisions concerning the confidentiality of information submitted on thts form, see General Information portion of the instructions. 
Title 18 USC1001 makes it a criminal offense for any person knowingly and willingly to make to any Agency or Department of the United States any false, 
fictitious, or fraudulent statements as to any matter within its jurisdiction. __ 


RESPONSE DUE: 

Within 1 hour of the incident, submit Schedule 1 and lines M - Q in Schedule 2 as an Emergency Alert report if criteria 1-8 are met. 

Within 6 hours of the incident, submit Schedule l and lines M - Q in Schedule 2 as a Normal Report if only criteria 9-12 are met. 

By the later of 24 hours after the recognition of the incident OR by die end of the next business day submit Schedule 1 & lines M - Q in Schedule 2 as a System 
Report if criteria 13-24 are met. Note 4 00pm local time will be considered the end of the business day 

Submit updates as needed andfor a final report (all of Schedules 1 and 2) within 72 hours of the incident. 

For NERC reporting entities registered in the United States; NERC has approved that the form OE-417 meets the submittal requirements for NERC. There may 
be other applicable regional, state and local reporting requirements._____ 


METHODS OF FILING RESPONSE 
(Retain a completed copy of this fonn for your files.) 

Online: Submit form via online submission at: https://www.oe.netl.doe.gov/OE417/ 

FAX: FAX Form OE-417 to the following facsimile number (202) 586-8485, 

Alternate: If you are unable to submit online or by fax, forms may be e-mailed to doehaeoc@liq.doe.gov, or call and report the information to the 

following telephone number (202) 586-8100.___ 


SCHEDULE 1 - ALERT CRITERIA 

age 1 of4) _ 


Criteria for Filing (Check all that apply) 

See Instructions For More Information _ 


1. [ ] Physical attack that causes major interruptions or impacts to critical infrastructure facilities or to operations 


EMERGENCY ALERT 
File within 1-Hour 

If any box 1-8 on the right is 
checked, tins fonn. must be 
filed within 1 hour of die 
incident; check Emergency 
Alert (for the Alert Status) on 
Line A below. 


2. [X] Cyber event that causes interruptions of electrical system operations 

3. [ ] Complete operational failure or shut-down of the transmission and/or distribution electrical system 

4. [ ] Electrical System Separation (Islanding) where part or parts of a power grid remain(s) operational in an otherwise 

blacked out area or within the partial failure of an integrated electrical system 

5. ( ] Uncontrolled loss of 300 Megawatts or more of firm system loads for 15 minutes or more from a single 

incident 

6. [ ] Finn load shedding of 100 Megawatts or more implemented under emergency operational policy 

7. [ ] System-wide voltage reductions of 3 percent or more 

8. [ ] Public appeal to reduce the use of electricity for purposes of maintaining the continuity of the Bulk Electric System 


NORNL4L REPORT 
File within 6-Hours 

If any box 9-12 on the right is 
checked AND none of die 
boxes 1 -S are checked, this 
form must be filed within 6 
hours of the incident; check 
Nonna! Report (for the Alert 
Status) on Line A below. 


9. [ ] Physical attack that could potentially impact electric power system adequacy or reliability, or vandalism which 

targets components of any security systems 

10. [XI Cyber event that could potentially impact electric power system adequacy or reliability 

11. [ ] Loss of electric service to more than 50,000 customers for 1 hour or more 

12. [ ] Fuel supply emergencies that could impact electric power system adequacy or reliability 




















3IOIZAJ ! V 3.JO.W riV! juui i iiueu uj 



SCHEDULE 1 - ALERT CRITERIA - CONTINUED 

{Page 2 of 4) 


13. [ ] Damage or destruction of a Facility within its Reliability Coordinator Area, Balan cin g Authority Area or 

Transmission Operator Area that results in action(s) to avoid a Bulk Electric System Emergency. 

14. [ ] Damage or destruction of its Facility' that results from actual or suspected intentional human action. 

15. [ ] Physical threat to its Facility excluding weather or natural disaster related threats, which has the potential to 

degrade the normal operation of the Facility. Or suspicious device or activity at its Facility. 

16. [ ] Physical threat to its Bulk Electric System control center, excluding weather or natural disaster related threats, which 

has the poteniinl to degrade the normal operation of the control center. Or suspicious device or activity at its Bulk 
Electric System control center. 

17. [ ] Bulk Electric System Emergency resulting in voltage deviation on a Facility; A voltage deviation equal to or 

greater than 10% of nominal voltage sustained for greater than or equal to 15 continuous minutes. 

18. [ ] Uncontrolled loss of 200 Megawatts or more of firm system loads for 15 minutes or more from a single incident for 

entities with previous year's peak demand less than or equai to 3,000 Megawatts 

19. [ J Total generation loss, within one min ute of: greater than or equal to 2,000 Megawatts in the Eastern or Western 

Interconnection or greater than or equal to 1,400 Megawatts in the ERCOT Interconnection. 

20. [ ] Complete loss of off-site power (LOOP) affecting a nuclear generating statical per the Nuclear Plant Interface 

Requirements. 

21. ( ] Unexpected Transmission loss within its area, contrary to design, of three or more Bulk Electric System 

Facilities caused by a common disturbance (excluding successful automatic reclosing). 

22. [ ] Unplanned evacuation from its Bulk Electric System control center facility for 30 continuous minutes or more. 

23. [ ] Complete loss of Interpersonal Communication and Alternative Interpersonal Communication capability affecting 

its staffed Bulk Electric System control center for 30 continuous minutes or more. 

24. [ ] Complete loss of monitoring or control capability at its staffed Bulk Electric System control center for 30 

continuous minutes or more. 


If significant changes have occurred after filing the initial report, re-file the form with the changes and check Update (for the Alert Status) on Line A below. 
The form must be re-filed within 72 hours of the incident with the latest information and Final (Alert Status) checked on Line A below, unless updated 


SYSTEM REPORT 
File within I-Business Bay 

If any box 13-24 on the right is 
checked AND none of the 
boxes 1-12 are checked, this 
form must be filed by the later 
of 24 hours after the 
recognition of the incident OR 
by the end of the next business 
day. Note 4:00pm local time 
will be considered the end of 
the business day. Check 
System Report (for the Alert 
Status) on Line A below. 



Alert Status (check one) 



Normal Report 

System Report 

Update 

[ ] 

[ 1 

[X] 

6 Hours 

1 Business Day 

As required 


Final 
[ ] 
72 Hours 


B. 

Organization Name 

sPower 

C. 

Address of Principal Business Office 

2180 South 1300 East Suite 600 Salt Lake City Utah 84106 
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SCHEDULE 1 


ALERT NOTICE 

3 of 4) 


D. 

Geographic Area(s) Affected 
(Coxaity, State) 

E. 

Date/Time Incident Began 
(min-dd yy/hhimm) using 24-hour clock 

F. 

Date/Time Incident Ended 
(mm-dd-yy/ hh mm) using 24-hour clock 

G. 

Did the incident/disturbance originate in your 
system/area? (check one) 

H. 

Estimate of Amount of Demand Involved 
(Peak Megawatts) 

I. 

Estimate of Number of Customers Affected 


California: Kem County, Los Angeles County; Utah: Salt Lake County; Wyoming: Converse County; 


m - os_ - ?mg ! na : _jl?__ 

mo dd yy hh mm 


[ ] Eastern [ ] Central [XI Mountain 

Pacific [ 1 Alaska [ 1 Hawaii 



No [ ) 

Unknown [X] 

Zero [XI 

Unknown [ ] 

Zero [XI 

Unknown [ ] 


SCHEDULE 1 


- TYPE OF EMERGENCY 

Check all that apply 


K. Impact 


L. Action Taken 


□ Unknown 

□ Physical attack 

□ Threat of physical attack 

□ Vandalism 

□ Theft 

□ Suspicious activity 

□ Cyber event (information technology) 

IS Cyber event (operational technology) 

□ Fuel supply emergencies, interruption, or 
deficiency 

□ Generator loss or failure not due to fuel supply 
interruption or deficiency or transmission 
failure 

□ Tr ansmi ssion equipment failur e (not including 
substation or switchyard) 

□ Failure at high voltage substation or switchyard 

□ Weather or natural disaster 

□ Operator action(s) 

□ Other 

IS Additional Infonnatiou/Coinments; 

Initial assessment revealed that a firewall exploit was Bidy uttced to 
execute a denial of service attack lhal caused the firewafis to reboot 
leading to an approximately 5 minute communications outage 


□ None 

□ Control center loss, failure, or evacuation 

□ Loss or degradation of control center monitoring 
or communication systems 

□ Damage or destruction of a fecility 

□ Electrical system separation (islanding) 

□ Complete operational failure or shutdown of the 
transmission and/or distribution system 

□ Major transmission system interruption (three or 
more BES elements) 

□ Major distribution system interruption 

□ Uncontrolled loss of 200 MW or more of firm 
system loads for 15 minutes or more 

□ Loss of electric service to more than 50,000 
customers for 1 hour or more 

□ System-wide voltage reductions or 3 percent or 
more 

□ Voltage deviation on an individual fecility of 
>10% for 15 minutes or more 

□ Inadequate electric resources to serve load 

□ Generating capacity loss of 1,400 MW or more 

□ Generating capacity loss of2,000 MW or more 

O Complete loss of off-site power to a nuclear 

generating station 

C2 Other 

IS Additional hiformation/Comments: 

Firewall reboots resulted in brief commoracsbons outages 
(approximately 5 minutes) between field devices at sites and between 
the sites and spowefs Control Center 


□ None 

□ Shed Firm Load: Load shedding of 100 
MW or more implemented under 
emergency operational policy (manually 
or automatically via UFLS or remedial 
action scheme) 

□ Public appeal to reduce the use of 
electricity for the purpose of maintaining 
the continuity of the electric power 
system 

□ Implemented a warning, alert, or 
contingency plan 

□ Voltage reduction 

□ Shed Interruptible Load 

□ Repaired or restored 

□ Mitigation implemented 

IS Other 

IS Additional Information/Comments 

After learning of the potential cause of the reboot, sPower 

started testing and deployment of an update to remove the 

exploited vulnerability 
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SCHEDULE 2 - NARRATIVE DESCRIPTION 

(Page 4 of 4) 

Information on Schedule 2 will not be disclosed to the public to the extent that it satisfies the criteria for exemption under the Freedom ofInformation Act, e.; 
exemptions for confidential commercial information and trade secrets, certain information that could endanger the physical safety of an individual, or 

information designated as Critical Energy Infrastructure Information 


iiMMuBB SS 


Lucas Root 


msssmsmn 


Name 


Title . _ 


one Number 


FAX Number •= 


..- 

Pro ride a description of the incident and actions taken to resolve it. Include as appropriate, the cause of the incident/disturbance, change in frequency, 
mitigation actions taken, equipment damaged, critical infrastructures interrupted, effects on other systems, and preliminary results from any 
investigations. Be sure to identify: the estimate restoration date, the name of any lost high voltage substations or switchyards, whether there was any. 
electrical system separation (and. if there were, what the islanding boundaries were), and the name of the generators and voltage Hues that were.lost 
(shown hy capacity type and ■voltage size groupmg)..ff necessary, copy and attach additional sheets. Equivalent docu men ts, containing this information can 
be supplied to meet the requirement; this includes the NERC EOP-004 Disturbance Report. Along with the filing of Schedule 2, a final (updated) Schedule 1 
needs to be filed. Check the Final box on line A for Alert Status on Schedule 1 and submit this and the completed Schedule 2 no later than 72 hours 
after detection that a criterion was met. ■ ___ : • ... 


R- Narrative: L i;L;l 

spower is a Generator Owner and Generator Operator of wind and solar generation assets that are operated from a Control Center in Sait Lake City, UT.(b) (7)(E) 


These updates are ongoing. 


S. Estimated Restoration Date for all Affected Customers 
Who Can Receive Power 




Pioneer. Beacon 4, ABSR, DSR1. DSR2, Beacon 1, Elevation C, WABSR0, Bayshore A, Bayshore B, Bayshore C, Solverde, 
firewaB at sPower headquarters 


Select if you appiove of all of the information provided on the Form being submitted to the North America Electric 
Reliability Coipotabon (NERC) and/or the Electricity Information Sharing and Analysis Center (E-TSAC) 

NERC is an entity that is certified by the Federal Energy Regulatory Commission to establish and enforce leliabiitty 
standards for the bulk power system but that is not part of the Federal Government This information would be 
submitted to help folfili the respondent's requirements under NERC's reliability standards. 

If approval is given to alert NERC and/or E-ISAC the Form will be emailed to systemawareness@nerc.net and/or 
opcrations@etsac.com when it is submitted to DOE. DOE is not responsible for ensuring the receipt of these emails 

by NERC and/or E -IS AC. 

8 Notify NERC | 85 Notify E-ISAC 


























Document 3 


From: 

Tarduoano. Matthew 

To: 

Evans. Karen S; Lotto, Adrienne 

Cc: 

Kenneth Buell (Kenneth.8uell@ha.doe.Dov): Kumar, Puesh 

Subject: 

Follow-Up on Denial of Service Incident 

Date: 

Friday, March 08, 2019 5:59:00 PM 


OFFICIAL USE ONLY 
Ail 

I wanted to quickly follow-up on the denial-of-service (DoS) incident experienced by sPower, a 
company that owns and operators soiar and wind generation assets in several states, that we noted 
this morning. 1 spoke with the operations manager of sPower this morning and the E-ISAC has also 
been in contact with the company. 

On March 5, sPower noted a series of brief communication outages between their control center 
and their remote sites. Upon further investigation, the communications outages were determined to 
have been caused by firewall reboots. After consulting with Cisco, the firewall manufacture, it was 
determined that the firewall reboots were caused by a DoS attack exploding a known vulnerability. 
Cisco recommended a firmware update, which sPower has been deploying across their system, after 
testing for compatibility. The DoS incdent was observed intermittently over a 12 hour period and no 
further activity has been observed since this time. sPower has reviewed log files and has found no 
evidence of a breach beyond the DoS attack. Additionally, the incident did not have any impacts on 
operations. 

When I spoke with sPower they did not expect to need any additional support or have any ongoing 
concerns; however, they will contact us if there are any issues, (b) (7)(E) 


We have been providing DOE's Office of Intelligence and 
Counterintelligence with updates and they are ready to investigate any indicators, as appropriate, 
and have been checking for any related incidents. We will work with IN and the ISACs to provide 
appropriate information to the sector as available. 

Please let me know if you have any questions. 

Best Regards, 

-Matt 

Matthew T. Tarduogno 

infrastructure Security & Energy Restoration 

Office of Cybersecurity, Energy Security, & Emergency Response 

U.S. Department of Energy 

Office: 202-586-2892 | Mobile: (b) (6) 

Matthew.Tarduogno@hq.doe.gov 
OFFICIAL USE ONLY 
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U.S. Department of Energy 
Electricity Delivery and 
Energy Reliability 
Form OE 417 


NOTICE: This report is mandatory under Public law 91 275 Failure to comply may result in criminal fines, civil penalties and other sanctions as provided by 
law For the sanctions and the provisions concerning, the confidentiality of information submitted on this form, sec General Information portion of the instructions 
Title 18 USC1001 makes it a criminal offense for any person knowingly and willingly to make to any Agency or Department of the United States any false, 
fictitious, or fraudulent statements 3S to any matter within its jurisdiction. _ 


RESPONSE DUE: 

Within 1 hour of the incident- submit Schedule 1 and lines M - Q in Schedule 2 as an Emergency Alert report if criteria 1-S are met. 

Within 6 hours of the incident, submit Schedule 1 and lines M - Q in Schedule 2 as a Normal Report if only criteria 9-12 are met 

By the later of 24 hours after the recognition of the incident QR by the end of the next business day submit Schedule 1 & lines M - Q in Schedule 2 as a System 
Report if criteria 13-24 are met. Note 4 00pm local time will be considered the end of the business day 

Submit updates as needed and/or a final report (all of Schedules I and 2) within 72 hours of the incident. 

For NERC reporting entities registered in the United States; NERC has approved that the form OEA17 meets the submittal requirements fin NERC. There may 
be other applicable regional, state and local reporting requirements.____ 


METHODS OF FILING RESPONSE 
(Retain a completed copy of this form for your files.) 


Online: Submit form via online submission at: https://www.oe.netl doe.gov/QE417/ 

FAX: FAX Fonn OE-417 to the following facsimile number: (202) 586-8485. 

Alternate: If you are unable to submit online or by fax, forms may be e-maiied to doehqeoc@hq doe. gov. or call and report the information to the 

following telephone number: (202) 586-8100.___ 


SCHEDULE 1 - ALERT CRITERIA 

age 1 of 4) 


Criteria for Filing (Check all that apply) 

See Instrncrions For More Information _ 


1. [ ] Physical attack that causes major interruptions or impacts to critical infrastructure facilities or to operations 


EMERGENCY ALERT 
File within 1-Hour 

If any box 1-8 on the right is 
checked, this fonn must be 
filed within l hour of the 
incident; check Em e r gency 
Alert (for the Alert Status) on 
Line A below. 


2. [X] Cyber event that causes interruptions of electrical system operations 

3. [ ] Complete operational failure or shut-down of the transmission and/or distribution electrical system 

4. [ ] Electrical System Separation (Islanding) where part or parts of a power grid remain(s) operational in an otherwise 

blacked out area or within the partial failure of an integrated electrical system 

5. [ ] Uncontrolled loss of 300 Megawatts or more of firm system loads for 15 minutes or more from a single 

incident 

6. [ ] Firm load shedding of 100 Megawatts at more implemented under emergency operational policy 

7. [ ] System-wide voltage reductions of 3 percent or more 

8. [ ] Public appeal to reduce the use of electricity for purposes of mai n taining the continuity of the Bulk Electric System 


NORMAL REPORT 
File within 6-Hours 

If any box 9-12 on the right is 
checked AND none of the 
boxes 1-8 are checked, this 
fonn must be filed within 6 
hours of the incident; check 
Normal Report (for the Alert 
Status) on Line A below. 


9. [ ] Physical attack that could potentially impact electric power system adequacy or reliability; or vandalism which 

targets components of any security systems 

10. [XI Cyber event that could potentially impact electric power system adequacy or reliability 

11. [ ] Loss of electric service to more than 50,000 customers for 1 hour or more 

12. [ ] Fuel supply emergencies that could impact electric power system adequacy or reliability 
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SYSTEM REPORT 
File within 1-Business Day 

If any box 13-24 on the fight is 
checked AND none of the 
boxes M2 are checked, this 
form must be filed by the later 
of 24 hours after the 
recognition of the incident OR 
by the end of the next business 
day. Note 4:00pm local tune 
will be considered the end of 
the business day. Cheek 
System Report (for the Alert 
Status) on Line A below. 


SCHEDULE 1 - ALERT CRITERIA - CONTINUED 

(Page 2 of 4) 


1 3. [ ] D ama ge or destruction of a Facility within its Reliability Coordinator Area, Balancing Authority Area or 

Transmission Operator Area that results in action(s) to avoid a Bulk Electric System Emergency. 

14. [ ] Damage or destruction of its Facility that results from actual or suspected intentional human action. 

15. [ ] Physical threat to its Facility secluding weather or natural disaster related threats, which has the potential to 

degrade the normal operation of the Facility. Or suspicious device or activity at its Facility. 

16. [ ] Physical threat to its Bulk Electric System control center, excluding weather or natural disaster related threats, which 

has the potential to degrade the normal operation of the control center. Or suspicious device or activity at its Bulk 
Electric System control center. 

17. [ ] Bulk Electric System Emergency resulting in voltage deviation on a Facility; A voltage deviation equal to or 

greater than 10% of nominal voltage sustained for greater than or equal to 15 continuous minutes. 

18. [ ] Uncontrolled loss of 200 Megawatts or more of firm, system loads for 15 minutes or more from a single incident for 

entities with previous year’s peak demand less than or equal to 3,000 Megawatts 

19. [ ] Total generation loss, within one minu te of greater than or equal to 2,000 Megawatts in the Eastern or Western 

Interconnection or greater than or equal to 1,400 Megawatts in the ERCOT Interconnection. 

20. [ ] Complete loss of off-site power (LOOP) affecting a nuclear generating station per the Nuclear Plant Interface 

Requirements. 

21. [ ] Unexpected Transmission loss within its area, contrary to design, of three or more Bulk Electric System 

Facilities caused by a common disturbance (excluding successful atttomatic reclosing). 

22. [ ] Unplanned evacuation from its Bulk Electric System control center facility for 30 continuous minutes or more. 

23. [ ] Complete loss of Interpersonal Communication and Alternative Interpersonal Communication capability affecting 

its staffed Bulk Electric System control center for 30 continuous minutes or more. 

24. [ ] Complete loss of monitoring or control capability at its staffed Bulk Electric System control center for 30 

continuous minutes or more. 


If significant changes have occurred after filing the initial report, re-file the form with the changes and check Update (for the Alert Status) on Line A below. 
The form must be re-filed within 72 hours of the incident with the latest information and Final (Alert Status) checked on Line A below, unless updated 


A. Alert Status (check one) 


B. Organization Name 


Address of Principal Business Office 




Normal Report 

System Report 

Update 

[ I 

[ I 

[ I 

6 Hours 

1 Business Day 

As required 


2180 South 1300 East Suite 600 Salt Lake City Utah 84106 


Final 

[XI 

72 Hours 
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ELECTRIC EMERGENCY INCIDENT AND 
DISTURBANCE REPORT 


SCHEDULE 1 - ALERT NOTICE 

3 of 4 


OMB Noi49Gl-0288 
Approval Expires; 05/31/2021 
Burden Per Response: 1.8 hour 


Geographic Axea(s) Affected 
(County. State 


California: Kem County, Los Angeles County; Utah: Salt Lake County; Wyoming: Converse County; 




Date/Time Incident Ended 
(mm-dd-w/ hhanm) using 24-hour clock 


Did the incident/disturbance originate in your 
system/area? (check one) 


Estimate of Amount of Demand Involved 
(Peak Megawatts) 


Estimate of Number of Customers Affected 


03 - Q5 - 2019 / 18 : 

mo dd yv hh r 


Yes [ ] 



No [ j 

Unknown [Xl 

z«o[Xl 

Unknown [ 3 

Zero [X] 

Unknown [ j 


J. Cause 


SCHEDULE 1 - TYPE OF EMERGENCY 

Check all that apply 


K. Impact 


L. Action Taken 


□ Unknown 

□ Physical attack 

□ Threat of physical attack 

□ Vandalism 

□ Theft 

□ Suspicious activity 

□ Cyber event (information technology) 

IS Cyber event (operational technology) 

□ Fuel supply emergencies, interruption, or 
deficiency 

□ Generator loss or failure not due to fuel supply 
interruption or deficiency or transmission 
failure 

□ Transmission equipment failure (not including 
substation or switchyard) 

□ Failure at high voltage substation or switchyard 

□ Weather or natural disaster 

□ Operator acfion(s) 

□ Other 

IS Additional fijfonnation/Comments: 

Initial assessmonl revealed that a firewall exploit was fiXely utilized to 
execute a denial of service attack that caused tie firewnis to reboot 
leading to an approximately 5 minute or less commuiscatJons outage 


□ None 

□ Control center loss, failure, or evacuation 

□ Loss or degradation of control center monitoring 
or communication systems 

□ Damage or destruction of a facility 

□ Electrical system separation (islanding) 

□ Complete operational failure or shutdown of the 
transmission and/or distribution system 

□ Major transmission system interruption (three or 
more BES elements) 

□ Major distribution system interruption 

□ Uncontrolled loss of 200 MW or more of firm 
system loads for 15 minutes or more 

□ Loss of electric service to more than 50,000 
customers for 1 hour or more 

□ System-wide voltage reductions or 3 percent or 
more 

□ Voltage deviation on an individual facility of 
>10% for 15 minutes or more 

□ Inadequate electric resources to serve load 

□ Generating capacity loss of 1,400 MW or more 

□ Generating capacity loss of2,000 MW or more 

□ Complete loss of off-site power to a nuclear 
generating station 

53 Other 

IS Additional htfonnation/Comments: 

Firewall reboots resulted in brief eorrernnkafions outages 
{approximately 5 minutes or less) between field devices at site3 and 
between the sites and sPower's Control Center 


□ None 

□ Shed Firm Load: Load shedding of 100 
MW or more implemented under 
emergency operational policy (manually 
or automatically via UFLS or remedial 
action scheme) 

□ Public appeal to reduce the use of 
electricity for the purpose of maintaining 
the continuity of the electric power 
system 

□ Implemented a warning, alert, or 
contingency plan ' 

□ Voltage reduction 

□ Shed Interruptible Load 

□ Repaired or restored 

□ Mitigation implemented 

53 Other 

IS Additional Infomiation/Comments 

After learning of the potential cause of the reboot, sPower 

started testing and deployment of an update to remove the 

exploited vulnerability 
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SCHEDULE 2 - NARRATIVE DESCRIPTION 

(Page 4 of 4) 

Information on Schedule 2 will not be disclosed to the public to the extent that it satisfies the criteria for exemption under the Freedom of Information Act, e,g., 
exemptions for confidential commercial information and trade secrets, certain information that could endanger the physical safety of an individual, or 

information designated as Critical Energy Infrastructure Information. 


NAME OF OFFICIAL THAT SHOULD BE CONTACTED FOR FOLLOW-UP OR ANY ADDITIONAL INFORMATION 


Lucas Root 


erations 






FAX Number 


E-mail Address 


Provide a description of the incident and actions taken to resolve it. Include as appropriate, the cause of theincident/distorbancc, change in frequency, 
mitigation actions taken, equipment damaged, critical infrastructures interrupted, effects on other systems, and preliminary’ results from any 
investigations. Be sore to identify’: the estimate restoration date, the name of any lost high voltage substations or switchyards, whether there was any 
electrical systemseparation (and if there were, what the islanding boundaries were), and the name of the generators and voltage lines tha t were lost 
(shown by capacity type and voltage size grouping). If necessary’, copy and attach additional sheets. Equivalent documents, contaimngtbis information ran 
be supplied to meet the requirement; this includes die NERC EOP-004 Disturbance Report. Along with the filing of Schedule 2, a final (updated) Schedule 1 
needs to be filed. Check the Final box on line A for Alert Status on Schedule 1 and submit this and the completed Schedule 2 no later than 72 hours 
after-detection that a criterion was met. 


Rl Narrative: 

spower is a Generator Owner and Generator Operator of wind and solar generation assets that are operated from a Control Center in Salt Lake City, UT: (fo) (7)(E) 


S. Estimated Restoration Date for all Affected Customer s 
Who Can Receive Power 


T. Name of Assets Impacted 



Pioneer, Beacon 4, ABSR, DSR1, DSR2, Beacon 1, Elevation C, WABSRB, Bayshore A, Bayshore B, Bayshore C, Sdverde, 
ftrewaB at sPowerheadquarters 



Select if you approve of all of the information provided on the Farm being submitted to the North America Electric 
Reliability Corporation (NERC) and/or the Electricity Information Sharing and Analysis Center (E-TSAQ 

NERC is an entity that is certified by Energy Regulatory Commission to establish and enfbreereliability 

standards for die bulk power system but that is not part of the Federal Government. This information would be 


submitted to help fulfill the respondent’s requirements under NERC's reliability standards. 

U. Notify NERC/E ISAC 



If approval ts given to alert NERC and/or E-ISAC the Form will be emailed to systemawaieness@nerc.net and/or 
operations@eisac.coni when it is submitted to DOE. DOE is not responsible for ensuring the receipt of these emails 


by NERC and/or E-ISAC 


8 Notify NERC | @5 Notify E-IS AC 



































